Image processing apparatus, image processing method and image processing program

ABSTRACT

A hemorrhage edge candidate area extraction section extracts a candidate area for the outline part of a hemorrhage area, based on an image signal of a medical image constituted by multiple color signals obtained by capturing an image of a living body. A feature quantity calculation section calculates a feature quantity of the hemorrhage area based on calculation of the amount of change in the image signal in a small area including the candidate area, among multiple small areas obtained by dividing the medical image. A hemorrhage edge determination section determines whether or not the candidate areas are the outline part of the hemorrhage area based on the feature quantity.

TECHNICAL FIELD

The present invention relates to an image processing apparatus, an imageprocessing method and an image processing program for extracting ahemorrhage edge based on a medical image obtained by an endoscopeapparatus and the like.

BACKGROUND ART

In the medical field, observation and diagnosis of an internal organ ina body cavity with the use of medical equipment having animage-capturing function, such as X-ray, CT and MRI apparatuses, anultrasonic observation apparatus and an endoscope apparatus, are widelyperformed.

For example in the case of an endoscope apparatus, a long and thininsertion section is inserted into a body cavity; an image of theinternal organ in the body cavity acquired by an objective opticalsystem provided for the tip of the insertion section is captured byimage-capturing means such as a solid-state image capturing device; andan endoscope image of the internal organ in the body cavity is displayedon the monitor screen based on the image-capturing signal. An operatorperforms observation and diagnosis based on the endoscope imagedisplayed on the monitor screen.

Since the endoscope apparatus is capable of directly capturing an imageof digestive tract mucous membrane, the operator can observe variousfindings such as the color tone of mucous membrane, the shape of alesion shape and a fine structure on the surface of mucous membrane.

Recently, a capsule endoscope apparatus has been developed as medicalequipment having a new image-capturing function, the usefulness of whichcan be expected similarly to the endoscope apparatus. The capsuleendoscope apparatus captures images of the inside of a body cavity aftera subject to be examined swallows a capsule endoscope until it isdischarged to the outside of the body and sends image-capturing signalsto a receiver provided outside the body. As many as several hours arerequired for the capsule endoscope to pass through each of digest tractsof esophagus, stomach, duodenum, small intestine and large intestineinside the body cavity and be discharged to the outside of the body,after the subject to be examined swallows the capsule endoscope.

Assuming that if the capsule endoscope apparatus captures, e.g., 2(frames of) images per second and sends the images to the receiveroutside the body, then it takes 6 hours from the capsule endoscopeapparatus being swallowed to being discharged outside of the body, thecapsule endoscope apparatus would capture as many as 43200 pieces ofimages while advancing in through the body cavity.

Displaying all of the large number of images on an observation apparatusto perform observation and diagnosis would take 72 minutes, a relativelylong period of time, even if, e.g., ten images are displayed per second.Accordingly, a surgeon to observe the captured images for such length oftime would be quite problematically subject to a high burden of time.

In addition, final diagnosis in endoscopy using a capsule endoscopeapparatus or a typical endoscope apparatus largely relied on thesubjectivity of a doctor, thus problematically causing fluctuation inthe diagnosis quality. For this reason, to improve the quality of imagediagnosis and reduce the time for shadow-reading an endoscope-image, ithas been expected to realize computer-aided diagnosis (CAD) that allowsfor automatically detecting from an endoscope image a presence of alesion such as hemorrhage, reddening, abnormal blood vessel and polyp.

Computer-aided diagnosis (CAD) is realized by an endoscope diagnosissupporting apparatus. The endoscope diagnosis supporting apparatusprovides objective and numerical diagnosis assistance by presenting adoctor with whether or not an image to be diagnosed is categorized insuch observations by using threshold processing or a statistical orinstatistical identification apparatus using various characteristicsamounts calculated from a region of interest (ROI) in an image. Also,the endoscope diagnosis supporting apparatus reduces the burden of thedoctor in shadow-reading, by selecting and presenting an image doubtedto include a lesion.

Meanwhile, several approaches have been used to detect hemorrhage whichcan be caused by various pathological reasons. One of such approaches ispresented in the publication of the PCT WO 02/073507 A2 which proposes amethod for automatically detecting hemorrhage based on hue, saturation,and brightness of an observation target area in an endoscope image,using the above-described endoscope diagnosis supporting apparatus.

However, in the method described in the publication, the hue,saturation, and brightness values of the observation target area arecompared with hue, saturation, and brightness sample values of a normalmucosa which are preset in the endoscope diagnosis supporting apparatus,and then it is determined whether the observation target area is anormal mucosa or a hemorrhage portion by means of diversion from thesample values.

Thus, there was a problem that the determination results depended on thesample values preset in the endoscope diagnosis supporting apparatus.

Therefore, the present invention aims to provide an image processingapparatus, an image processing method, and an image processing programcapable of detecting a hemorrhage area by using an amount of change inan image signal or a color signal in an outline part of a bloody area.

DISCLOSURE OF INVENTION Means for Solving the Problem

An image processing apparatus of a first aspect of the present inventionincludes:

a hemorrhage edge candidate area extraction section for extracting acandidate area for the outline part of a hemorrhage area, based on animage signal of a medical image constituted by multiple color signalsobtained by capturing an image of a living body;

a feature quantity calculation section for calculating a featurequantity of the hemorrhage area based on calculation of the amount ofchange in the image signal in a small area including the candidate area,among multiple small areas obtained by dividing the medical image; and

a hemorrhage edge determination section for determining whether or notthe candidate area is the outline part of the hemorrhage area based onthe feature quantity.

An image processing apparatus of a second aspect of the presentinvention includes:

an evaluation area setting section for dividing a medical imagecorresponding to a medical image signal constituted by multiple colorsignals obtained by capturing an image of a living body, into multiplesmall areas, extracting a small area including the outline part of ahemorrhage area from the multiple small areas based on at least one ofthe color signals, setting the extracted small area as a hemorrhageevaluation area, and setting evaluation target areas constituted by themultiple small areas, around the hemorrhage evaluation area;

a hemorrhage candidate area determination section for extractinghemorrhage edge candidate areas from the evaluation target areas basedon the amount of change in the color signals in the evaluation targetareas and determining whether or not the hemorrhage evaluation area is ahemorrhage candidate area based on the ratio of the hemorrhage edgecandidate areas to the evaluation target areas; and

a hemorrhage area determination section for extracting the outline partof a hemorrhage area from the hemorrhage edge candidate areas based onchange in two or more of the color signals in the hemorrhage edgecandidate areas and determining whether or not the hemorrhage candidatearea is the hemorrhage area based on the ratio of the outline part ofthe hemorrhage area to the hemorrhage edge candidate areas.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically showing the network configuration ofan image processing apparatus according to a first embodiment of thepresent invention and related systems;

FIG. 2 is a diagram schematically showing the entire configuration ofthe image processing apparatus;

FIG. 3 is a flowchart illustrating the procedure for image analysisprocessing by an image processing program;

FIG. 4 is a flowchart illustrating the procedure for bleeding part edgecandidate extraction processing;

FIG. 5 is a schematic diagram for illustrating analysis data, wherein:FIG. 5( a) is a schematic diagram illustrating an original image; FIG.5( b) is a schematic diagram illustrating a bleeding part edge candidateimage; FIG. 5( c) is a schematic diagram illustrating a shape edgeimage; and FIG. 5( d) is a schematic diagram illustrating a bleedingpart edge image;

FIG. 6 is a flowchart illustrating the procedure for shape edgeextraction processing;

FIG. 7 is a diagram illustrating a method for calculating R changes;

FIG. 8 is a flowchart illustrating the procedure for bleeding part edgedetermination processing;

FIG. 9 is a flowchart illustrating the procedure for image analysisprocessing by an image processing program in a second embodiment of thepresent invention;

FIG. 10 is a flowchart illustrating the procedure for calculating acolor edge feature quantity;

FIG. 11 is a diagram illustrating a method for calculating R changes andG changes;

FIG. 12 is a flowchart illustrating the procedure for image analysisprocessing by an image processing program in a third embodiment of thepresent invention;

FIG. 13 is a schematic diagram illustrating the relation between ableeding part edge candidate area and a background area;

FIG. 14 schematically shows the values of R signals of pixels positionedon the C-C′ line in FIG. 13; and FIGS. 14( a) and 14(b) show change inthe value of the R signal in the case where the bleeding part edgecandidate area is a bleeding part edge and change in the value of the Rsignal in the case where the bleeding part edge candidate area is anedge formed by an element other than bleeding, respectively;

FIG. 15 is a flowchart illustrating the procedure for image analysisprocessing by an image processing program in a fourth embodiment of thepresent invention;

FIG. 16 is a schematic diagram illustrating the relation between ableeding part edge candidate area and an internal area;

FIG. 17 is a diagram schematically showing the entire configuration ofan image processing apparatus of a fifth embodiment;

FIG. 18 is a flowchart illustrating the procedure for image analysisprocessing by an image processing program;

FIG. 19 is a flowchart illustrating the procedure for edge extractionprocessing;

FIG. 20 is a schematic diagram illustrating a method for calculating Gchanges;

FIG. 21 is a flowchart illustrating the procedure for bleeding partcandidate extraction processing;

FIG. 22 is a diagram illustrating the bleeding part candidate extractionprocessing;

FIG. 23 is a diagram illustrating a method for determining bleeding partcandidates;

FIG. 24 is a flowchart illustrating the procedure for bleeding partdetermination processing;

FIG. 25 is a diagram illustrating a method for calculating R changes andG changes;

FIG. 26 is a flowchart illustrating the procedure for bleeding part edgecandidate determination processing in the second embodiment;

FIG. 27 is a diagram illustrating positions where peripheral areas areset in the fifth embodiment;

FIG. 28 is a diagram illustrating bleeding part candidate determinationprocessing in the fifth embodiment; and

FIG. 29 is a diagram illustrating a pattern of arrangement of peripheralareas in a sixth embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described below withreference to drawings.

First Embodiment

A first embodiment of the present invention will be described below withreference to FIGS. 1 to 8. The object of this embodiment is to providean image processing apparatus, an image processing method and an imageprocessing program capable of relatively determining and detectingwhether or not an area is a hemorrhage edge or not with the use of theamount of change in color signals at the outline part of a hemorrhagearea (hereinafter referred to as a hemorrhage edge).

As examples of the hemorrhage area, a bleeding part area where bleedingfrom mucous membrane actually occurs, a reddened part area where thesurface of mucous membrane is reddened by hyperemia, and the like aregiven. In this embodiment, the case of detecting, for example, theoutline part of a bleeding part area (hereinafter referred to as ableeding part edge) will be described.

First, description will be made on the network configuration formed byan image processing apparatus 1 according to the first embodiment andits related systems, based on FIG. 1. FIG. 1 is a diagram schematicallyshowing the network configuration formed by the image processingapparatus 1 according to the first embodiment and its related systems.

As shown in FIG. 1, the image processing apparatus 1 which performsvarious image processings and information processings is connected to aLAN 2 which uses TCP/IP as its communication protocol. Meanwhile, anendoscope observation apparatus 3 which captures an image of mucousmembrane or the like in a living body and outputs an image signal of amedical image is also connected to the LAN 2 via an endoscope filingapparatus 4.

The endoscope filing apparatus 4 receives or acquires an image signalfrom the endoscope observation apparatus 3 to generate image data, andaccumulates the generated image data. Then, the image processingapparatus 1 acquires the image data accumulated in the endoscope filingapparatus 4 via the LAN 2.

Next, the entire configuration of the image processing apparatus 1 willbe described. The image processing apparatus 1 is configured with ageneral-purpose personal computer 11 as its center and provided with anoperation device 12 configured by a keyboard and a mouse, a storagedevice 13 configured by a hard disk and a display device 14 configuredby a CRT.

FIG. 2 is a diagram schematically showing the entire configuration ofthe image processing apparatus 1. Each of the operation device 12, thestorage device 13 and the display device 14 is electrically connected tothe personal computer 11. Specification of image data to be processed,acquisition and display of the specified image data and instruction toexecute processing are inputted from the operation device 12. The resultof various processings performed by the image processing apparatus 1 isdisplayed on the display device 14.

The personal computer 11 has a CPU 21 which performs execution andcontrol of various programs, a memory 22 which stores the variousprocessing programs and data, an external storage I/F 23 which performswriting and reading of information to and from the storage device 13, anetwork card 24 which communicates information with external equipment,an operation I/F 25 which receives an operation signal inputted from theoperation device 12 and performs necessary data processing, and agraphic board 26 which outputs a video signal to the display device 14.The CPU 21, the memory 22, the external storage I/F 23, the network card24, the operation I/F 25 and the graphic board 26 which constitute thepersonal computer 11 are electrically connected with one another via abus 27. Therefore, these components in the personal computer 11 canmutually send and receive information via the bus 27.

The network card 24 is electrically connected to the LAN 2 and sends andreceives information to and from the endoscope filing apparatus 4 whichis also connected to the LAN 2.

The external storage I/F 23 reads an image processing program 28 storedin the storage device 13 and stores it in the memory 22. The imageprocessing program 28 is a program which executes image analysisprocessing and configured by multiple execution files, dynamic linklibrary files or setting files. By the image processing program 28 whichis stored in the memory 22 being executed, the CPU 21 operates.

The CPU 21 performs image analysis processing, such as detection anddetermination of the outline part (edge part) of a hemorrhage area, tobe described later for image data acquired from the endoscope filingapparatus 4. Analysis data 29 acquired or generated by each processingby the CPU 21 is stored in the memory 22. This analysis data 29 includesan original image 31 which is the image data acquired from the endoscopefiling apparatus 4.

Furthermore, the analysis data 29 includes a bleeding part edgecandidate image 32, a shape edge image 33 and a bleeding part edge image34 generated by various processings to be described later. Each of theseimages 32 to 34 will be described later in detail. The CPU 21 hasfunctions in FIG. 3 to be described below, that is, a bleeding part edgecandidate extraction function, a feature quantity calculation functionand a bleeding part edge determination function.

The operation of the image processing apparatus 1 configured asdescribed above will be described. In this embodiment, the case ofdetecting, for example, the outline part of a bleeding part area(hereinafter referred to as a bleeding part edge) will be described withthe use of the flowchart in FIG. 3.

FIG. 3 is a flowchart illustrating the procedure for image analysisprocessing by the image processing program 28. First, at step S1, whichis an original image acquisition step, the CPU 21 acquires image dataspecified by the operation device 12 from the endoscope filing apparatus4 and stores it in the memory 22 as the original image 31. The originalimage 31 is a color image constituted by three planes of red (R), green(G) and blue (B). The tone number of a pixel of each plane takes an8-bit value, that is, a value between 0 and 255.

Next, at step S2, which is an image analysis processing step, the CPU 21performs various processings for the original image 31 acquired at stepS1 to generate the bleeding part edge candidate image 32, the shape edgeimage 33 and the bleeding part edge image 34.

This image analysis processing step (step S2) is constituted by bleedingpart edge candidate extraction processing (step S10) for generating thebleeding part edge candidate image 32 from the original image 31, shapeedge extraction processing (step S20) for generating the shape edgeimage 33 from the bleeding part edge candidate image 32 based on featurequantity calculation processing, and bleeding part edge determinationprocessing (step S40) for generating the bleeding part edge image 34from the bleeding part edge candidate image 32 and the shape edge image33.

There is a possibility that the shape edge image 33 and the bleedingpart edge candidate image 32 are generated being mixed with each other,by the feature quantity calculation processing, and the bleeding partedge image 34 is generated by bleeding part edge processing.

The processings of the steps S10, S20 and S40 are executed in thatorder. Each of the processings in the image analysis processing step(step S2) will be described in accordance with the processing order.

First, the bleeding part edge candidate extraction processing at stepS10 will be described with the use of FIG. 4. FIG. 4 is a flowchartillustrating the procedure for the bleeding part edge candidateextraction processing. First, at step S11, the CPU 21 divides theoriginal image 31 into N×N small areas. In this embodiment, for example,N=36 is set.

Reverse gamma correction or shading correction for the original image 31may be added as preprocessing before step S11. In this case, thecorrected image for which the correction processing has been performedis divided into small areas, and subsequent processings are performedtherefor.

Next, at step S12, the CPU 21 initializes i which indicates a numberidentifying the divided small area (hereinafter referred to as a dividedarea) to be analyzed to 1. The number i identifying the divided area 43is an integer value equal to or above 1 and equal to or below N×N.

Next, at step S13, the CPU 21 acquires the values (luminance values) ofR signals, G signals and B signals of the pixels included in the i-thdivided area 43 and calculates average values Ra_(i), Ga_(i) and Ba_(i)of the respective color signals in the i-th divided area 43.

Then, at step S14, the CPU 21 compares a bleeding part area color tonespace which has been set in advance and the average values Ra_(i),Ga_(i) and Ba_(i) of the respective color signals calculated at stepS13.

The bleeding part area color tone space is an internal space separatedinto the luminance level range of the R signal (Rmin≦x≦Rmax), theluminance level range of the G signal (Gmin≦y≦Gmax) and the luminancelevel range of the B signal (Bmin≦z≦Bmax) within which a bleeding partcan exist, in a three-dimensional space with the luminance levels of theR signal, the G signal and the B signal indicated by the X axis, the Yaxis and the Z axis, respectively.

At step S14, the CPU 21 determines whether or not the average valuesRa_(i), Ga_(i) and Ba_(i) of the respective color signals calculated atstep S13 exist within the bleeding part area color tone space. If theyexist, the CPU 21 proceeds to step S15.

That is, if the calculated average values Ra_(i), Ga_(i) and Ba_(i) ofthe respective color signals are within the bleeding part area colortone space, that is, if Rmin≦Ra_(i)≦Rmax, Gmin≦Ga_(i)≦Gmax andBmin≦Ba_(i)≦Bmax are satisfied, then the CPU 21 proceeds to processingat step S15. At this step S15, the CPU 21 determines that the i-thdivided area 43 is a bleeding part edge candidate.

On the other hand, if the average values Ra_(i), Ga_(i) and Ba_(i) ofthe respective color signals calculated at step S13 do not exist withinthe bleeding part area color tone space, that is, if any of Rmin>Ra_(i),Ra_(i)>Rmax, Gmin>Ga_(i), Ga_(i)>Gmax, Bmin>Ba_(i) and Ba_(i)>Bmax issatisfied, then the CPU 21 proceeds to processing at step S16. At thisstep S16, the CPU 21 determines that the i-th divided area 43 is not ableeding part edge candidate.

It is also possible to use a color tone space of G/R and B/G todetermine a bleeding part edge candidate. That is, if(G/R)min≦Ga_(i)/Ra_(i)≦(G/R)max and (B/G)min≦Ba_(i)/Ga_(i)≦(B/G)max aresatisfied, then the CPU 21 determines that the i-th divided area 43 is ableeding part edge candidate.

Here, (G/R)min and (G/R)max indicate the smallest value and the largestvalue of G/R, respectively. And, (B/G)min and (B/G)max indicate thesmallest value and the largest value of B/G, respectively.

When the processing at step S15 or S16 ends, the CPU 21 then determineswhether or not the bleeding part edge candidate determination has beenperformed for all the divided areas 43, at step S17.

Specifically, if i<N×N is satisfied, then the CPU 21 adds 1 to thenumber i identifying the divided area 43 (i=i+1) at step S18 and returnsto step S13 to perform the bleeding part edge candidate determinationprocessing for the remaining divided areas. If i=N×N is satisfied, theCPU 21 terminates the processing in FIG. 4 (that is, the bleeding partedge candidate extraction processing at step S10 in FIG. 3) and proceedsto the shape edge extraction processing at the subsequent step S20 inFIG. 3.

When the bleeding part edge candidate extraction processing ends, ableeding edge candidate image 32 as shown in FIG. 5( b) is generatedfrom an original image 31 as shown in FIG. 5( a). FIGS. 5( a) to 5(d)are schematic diagrams illustrating the analysis data 29.

That is, FIG. 5( a) is a schematic diagram illustrating an originalimage 31; FIG. 5( b) is a schematic diagram illustrating a bleeding partedge candidate image 32; FIG. 5( c) is a schematic diagram illustratinga shape edge image 33; and FIG. 5( d) is a schematic diagramillustrating a bleeding part edge image 34.

In the original image 31 in FIG. 5( a), there exist a mucous membraneshape area 41, such as a groove formed on the surface of mucousmembrane, and a bleeding part area 42. In the bleeding part edgecandidate image 32 shown in FIG. 5( b) which has been obtained based onthe original image 31, all divided areas 43 are classified as any ofbleeding part edge candidate areas 44 (shaded area) and non bleedingpart edge candidate areas 45, and the bleeding part edge candidate areas44 include the mucous membrane shape area 41 and the bleeding part area42.

Next, the shape edge extraction processing will be described with theuse of FIG. 6. FIG. 6 is a flowchart illustrating the procedure for theshape edge extraction processing at step S20 in FIG. 3.

In this shape edge extraction processing, the CPU 21 performs processingfor extracting the outline part of the mucous membrane shape area 41(hereinafter referred to as a shape edge) as a shape edge area formed bya large area in which multiple divided areas 43 are connected. First, atstep S21, the CPU 21 initializes which indicates the number identifyingthe divided area 43 to be analyzed, to 1. The number i identifying thedivided area 43 is an integer value equal to or above 1 and equal to orbelow N×N.

Next, at step S22, the CPU 21 calculates change in the value of the Rsignal (hereinafter referred to as R change) in the i-th divided area43. The R change is calculated from the value of the R signal of aparticular pixel (R1) in the i-th divided area 43 and the value of the Rsignal of a different particular pixel (R2) in the same divided area 43.Specifically, the CPU 21 performs the calculation with the formula of Rchange=log_(e)(R2)−log_(e)(R1).

In this embodiment, the CPU 21 calculates the R change for each of eightdirections, that is, upward and downward directions, right and leftdirections and diagonal directions in the divided area 43, as shown inFIG. 7. FIGS. 7( a) to 7(h) are diagrams illustrating a method forcalculating the R changes.

First R change is upward R change as shown in FIG. 7( a), and it iscalculated with the value of the R signal of the pixel at the lowercenter as R1 and the value of the R signal of the pixel at the uppercenter as R2. Second R change is R change toward the diagonallyupper-right direction as shown in FIG. 7( b), and it is calculated withthe value of the R signal of the pixel at the lower left as R1 and thevalue of the R signal of the pixel at the upper right as R2. Third Rchange is R change toward the right direction as shown in FIG. 7( c),and it is calculated with the value of the R signal of the pixel at theleft center as R1 and the value of the R signal of the pixel at theright center as R2.

Fourth R change is R change toward the diagonally lower-right directionas shown in FIG. 7( d), and it is calculated with the value of the Rsignal of the pixel at the upper left as R1 and the value of the Rsignal of the pixel at the lower right as R2. Fifth R change is downwardR change as shown in FIG. 7( e), and it is calculated with the value ofthe R signal of the pixel at the upper center as R1 and the value of theR signal of the pixel at the lower center as R2. Sixth R change is Rchange toward the diagonally lower-left direction as shown in FIG. 7(f), and it is calculated with the value of the R signal of the pixel atthe upper right as R1 and the value of the R signal of the pixel at thelower left as R2.

Seventh R change is R change toward the left direction as shown in FIG.7( g), and it is calculated with the value of the R signal of the pixelat the right center as R1 and the value of the R signal of the pixel atthe left center as R2. Eighth R change is R change toward the diagonallyupper-left direction as shown in FIG. 7( h), and it is calculated withthe value of the R signal of the pixel at the lower right as R1 and thevalue of the R signal of the pixel at the upper left as R2.

Next, at step S23, the CPU 21 acquires the largest value among the firstto eighth R changes calculated at step S22 as an edge feature quantityA_(i). Next, at step S24, the CPU 21 determines whether or not the i-thdivided area 43 is an edge. Specifically, if the edge feature quantityA_(i)>Th1 is satisfied, then it is determined that the i-th divided area43 is an edge.

Here, Th1 is a threshold to be a reference used when edge determinationis performed, and, for example, Th1=0.14 is set in this embodiment. IfA_(i)>Th1 is satisfied, then the CPU 21 determines at step S25 that thei-th divided area 43 is an edge and proceeds to processing at step S27.If A_(i)≦Th1 is satisfied, the CPU 21 determines at step S26 that thei-th divided area 43 is not an edge and proceeds to processing at stepS27.

At step S27, the CPU 21 determines whether or not the edge determinationhas been performed for all the divided areas 43. Specifically, if i<N×Nis satisfied, then the CPU 21 adds 1 to the number i identifying thedivided area 43 (i=i+1) at step S28 and returns to step S22 to performthe edge determination at steps S22 to S26 for the remaining dividedareas 43. If i=N×N is satisfied, then the CPU 21 terminates the edgedetermination processing and proceeds to processing at step S29.

At step S29, the CPU 21 performs labeling processing for the areas whichhave been determined to be edges at step S25 among the N×N divided areas43. The labeling processing in this embodiment is performed as describedbelow.

The CPU 21 scans image data sequentially from the upper left to thelower right to find a divided area 43 which is not labeled and which hasbeen determined to be an edge area and give it a predetermined number asa label value.

In this case, the CPU 21 does not use a label value already given toanother divided area 43 and gives the divided area 43 a value obtainedby adding 1 to the largest value among already given label values.

If no divided area 43 is given a label value, then the CPU 21 gives, forexample, 1 to the divided area 43 as a label value. Next, the CPU 21gives the same label value to all such divided areas 43 as are connectedto the divided area 43 which has been given the label value and aredetermined to be an edge area.

The CPU 21 repeats the scanning and the label-value giving describedabove until a label value is given to all the divided areas 43determined to be an edge area. That is, the same label value is given todivided areas 43 belonging to the same connection part, and a differentlabel value is given to each connection part.

Next, at step S30, the CPU 21 acquires the largest value among the labelvalues given to the N×N divided areas 43 at step S29, as L.Subsequently, at step S31, the CPU 21 initializes j which indicates aconnected, divided area 43 to be analyzed, to 1. The label value jidentifying the connected, divided area 43 takes an integer value equalto or above 1 and equal to or below L. Next, at step S32, the CPU 21counts the number of divided areas 43 having the label value j andacquires the counted number of areas M_(j).

Subsequently, at step S33, the CPU 21 performs label classificationdetermination for determining whether the label value j is a shape edgelabel or a bleeding part edge label. Specifically, if the number ofareas M_(j)>Th2 is satisfied, then the CPU 21 determines the label valuej is a shape edge label. Here, Th2 is a threshold to be a referencevalue used for determining whether the label is a shape edge label or ableeding part edge label, and, for example, Th2=10 is set in thisembodiment. If M_(j)>Th2 is satisfied, then the CPU 21 determines atstep S34 that the label value j is a shape edge label and proceeds toprocessing at step S36. If M_(j)≦Th2 is satisfied, then the CPU 21determines at step S35 that the label value j is a bleeding part edgelabel and proceeds to the processing at step S36.

At step S36, the CPU 21 determines whether or not the labelclassification determination has been performed for all the dividedareas 43 determined to be edges.

Specifically, if j<L is satisfied, then the CPU 21 adds 1 to the labelvalue j identifying the connected, divided area 43 (j=j+1) at step S37and returns to step S32 to perform the label classificationdetermination at steps S32 to S35 for the remaining divided areas. Ifj=L is satisfied, the CPU 21 terminates the processing in FIG. 6 andthen proceeds to the bleeding part edge determination processing at stepS40 in FIG. 3.

When the shape edge extraction processing in FIG. 6 ends, a shape edgeimage 33 as shown in FIG. 5( c) is generated. In the shape edge image 33shown in FIG. 5( c), divided areas 43 with a label value classified asthe shape edge label are shown as shape edge areas 46 (shaded areas).The shape edge areas 46 correspond to the divided areas 43 correspondingto the outline part of the mucous membrane shape area 41 among thebleeding part edge candidate areas 44.

Next, the bleeding part edge determination processing at step S40 inFIG. 3 will be described with the use of FIG. 8. FIG. 8 is a flowchartillustrating the procedure for the bleeding part edge determinationprocessing.

In the bleeding part edge determination processing, the CPU 21 performsprocessing for determining bleeding part edge areas 47 among thebleeding part edge candidate areas 44 extracted by the bleeding partedge candidate extraction processing. First, at step S41, the CPU 21initializes i which indicates the number identifying the divided area 43to be analyzed, to 1.

The number i identifying the divided area 43 is an integer value equalto or above 1 and equal to or below N×N. Next, at step S42, the CPU 21determines whether or not the i-th divided area 43 is a bleeding partedge candidate area 44.

The determination processing at step S42 is performed based on theresult of the bleeding part edge candidate extraction processing at stepS10 in FIG. 3. If the i-th divided area 43 is a bleeding part edgecandidate area 44, then the CPU 21 proceeds to processing at step S43and determines whether or not the label value given to the i-th dividedarea 43 is the bleeding part edge label. If the i-th divided area 43 isnot a bleeding part edge candidate area 44, then the CPU 21 determinesthat the divided area is not the bleeding part edge area 47 and proceedsto step S46.

The determination at step S43 is performed based on the result of theshape edge extraction processing at step S20 in FIG. 3. If the labelvalue given to the i-th divided area 43 is a bleeding part edge label,then the CPU 21 determines at step S44 that the i-th divided area 43 isthe bleeding part edge area 47 and proceeds to step S46.

If the label value given to the i-th divided area 43 is not a bleedingpart edge label, that is, if the label value given to the i-th dividedarea 43 is a shape edge label or no label is given thereto, then the CPU21 determines at step S45 that the divided area is not the bleeding partedge area 47 and proceeds to step S46.

At step S46, the CPU 21 determines whether or not the bleeding part edgearea determination has been performed for all the divided areas 43.Specifically, if i<N×N is satisfied, then the CPU 21 adds 1 to thenumber i identifying the divided area 43 (i=i+1) at step S47 and returnsto step S42 to perform the bleeding part edge area determination for theremaining divided areas 43. If i=N×N is satisfied, then the CPU 21terminates the bleeding part edge determination processing in FIG. 8.

When the bleeding part edge determination processing ends, a bleedingpart edge image 34 as shown in FIG. 5( d) is generated. In the bleedingpart edge image 34 shown in FIG. 5( d), the outline part of the bleedingpart area 42 in the original image 31 shown in FIG. 5( a), that is, ableeding part edge is displayed as the bleeding part edge areas 47(shaded part).

Through the above processings, the image processing apparatus 1 canacquire the original image 31 captured by the endoscope observationapparatus 3 via the image filing apparatus 4 and detect a bleeding partedge in the original image 31.

As described above, in the image processing apparatus 1 of thisembodiment, it is possible to relatively determine and detect whether ornot a divided area is a bleeding part edge by using the amount of changein particular color signals in a bleeding part edge. Furthermore, inthis embodiment, since whether an edge is a shape edge or a bleedingpart edge is determined based on the size of the edge, it is possible toprecisely extract a bleeding part edge. Furthermore, according to thisembodiment, an operator can improve the quality of imaging diagnosis andshorten the time required for interpretation of an endoscope image byobserving a bleeding part edge image 34.

Second Embodiment

Next, a second embodiment of the present invention will be described. Inthe first embodiment described above, an edge area is extracted with theuse of an edge feature quantity A_(i), which is the largest value ofchange of an R signal, and it is determined whether or not the edge areais a bleeding part edge based on the size of the edge area. On the otherhand, in this embodiment, a color edge feature quantity B_(i) iscalculated based on change of two or more color signals among the Rsignal, the G signal and the B signal, and it is determined whether ornot an edge area is a bleeding part edge, with the use of the color edgefeature quantity B_(i).

The hardware configuration of the image processing apparatus 1 of thisembodiment is the same as that of the image processing apparatus 1 inFIG. 2. An image processing program 51 different from the imageprocessing program 28 stored in the storage device 13 in FIG. 2 isstored. Furthermore, the analysis data stored in the memory 22 includestwo kinds of image data, that is, the original image 31 and the bleedingpart edge image 34, unlike the case in FIG. 2.

That is, the image processing apparatus 1 of this embodiment is the sameas to that of the first embodiment except that the content of theprocessing performed by the image processing program 51 is different,and that analysis data 52 acquired, generated and stored in the memory22 by executing the image processing program 51 includes the two kindsof data, the original image 31 and the bleeding part edge image 34, anddoes not include the bleeding part edge candidate image 32 and the shapeedge image 33. Therefore, only characteristic operations will bedescribed in this embodiment. The same components will be given the samereference numerals, and description thereof will be omitted.

In this embodiment, processing for detecting, for example, a bleedingpart edge will be described with the use of the flowchart in FIG. 9,similarly to the first embodiment. FIG. 9 is a flowchart illustratingthe procedure for image analysis processing by the image processingprogram 51.

First, at step S110, the CPU 21 acquires image data specified by theoperation device 12 from the image filing apparatus 4 and stores it inthe memory 22 as an original image 31. Next, at step S120, the CPU 21divides the original image 31 acquired at step S110 to generate N×Ndivided areas 43. Next, at step S130, the CPU 21 initializes i whichindicates the number identifying the divided area 43 to be analyzed,to 1. The number i identifying the divided area 43 is an integer valueequal to or above 1 and equal to or below N×N.

Next, at step S140, the CPU 21 calculates the color edge featurequantity B_(i) in the i-th divided area 43. The procedure forcalculating the color edge feature quantity at step S140 will bedescribed with the use of the flowchart in FIG. 10. FIG. 10 is aflowchart illustrating the procedure for calculating the color edgefeature quantity.

First, at step S141, the CPU 21 calculates change in the value of the Rsignal (hereinafter referred to as R change) and change in the value ofthe G signal (hereinafter referred to as G change) in the i-th dividedarea 43. The R change is calculated from the value of the R signal (R1)of a particular pixel P1 in the divided area 43 and the value of the Rsignal (R2) of a different particular pixel P2 in the same divided area43. Specifically, the CPU 21 performs calculation with the formula of Rchange=log_(e)(R2)−log_(e)(R1).

The G change is calculated from the value of the G signal (G1) of thepixel P1 used when the R change is calculated and the value of the Gsignal (G2) of the pixel P2. Specifically, the G change is calculated bythe CPU 21 with G change=log_(e)(G2)−log_(e)(G1). In this embodiment,the CPU 21 calculate the R change and the G change for each of eightdirections, that is, upward and downward directions, right and leftdirections and diagonal directions in the divided area 43, as shown inFIG. 11. FIG. 11 is a diagram illustrating a method for calculating theR changes and the G changes. Since the method for calculating the firstto eighth R changes shown in FIG. 11 is similar to the method forcalculating the first to eighth R changes shown in FIG. 7, descriptionthereof will be omitted.

The method for calculating the first to eighth G changes shown in FIG.11 uses the same pixels used in calculation of the first to eighth Rchanges, and the method is similar to the method for calculating thefirst to eighth R changes if R1 and R2, the values of the R signal, aresubstituted with G1 and G2, the values of the G signal, respectively.Therefore, description of the method will be omitted.

Next, at step S142, the CPU 21 determines the first to eighth changeratios by dividing the first to eighth G changes by the first to eighthR changes, respectively. At the subsequent step S143, the CPU 21acquires the largest value among the first to eighth change ratiosdetermined at step S142 as a color edge feature quantity B_(i).

In the vicinity of a boundary area between peripheral mucous membraneand a bleeding part, that is, in the vicinity of a bleeding part edge,change in the G signal is generally larger than change in the R signalor the B signal. Therefore, in this embodiment, the largest value of Gchange/R change is assumed to be the color edge feature quantity B_(i).The CPU 21 may use the largest value of B change/R change as the coloredge feature quantity B_(i).

Next, at step S150 in FIG. 9, the CPU 21 determines whether or not thei-th divided area 43 is a bleeding part edge. Specifically, if the coloredge feature quantity B_(i)>Th3 is satisfied, then the CPU 21 determinesthat the i-th divided area 43 is a bleeding part edge.

Here, Th3 is a threshold to be a reference value used when it isdetermined that a divided area is a bleeding part edge, and, forexample, Th3=0.1 is set in this embodiment. If B_(i)>Th3 is satisfied,then the CPU 21 determines at step S160 that the i-th divided area 43 isa bleeding part edge and proceeds to processing at step S180.

If B_(i)≦Th3 is satisfied, then the CPU 21 determines at step S170 thatthe i-th divided area 43 is not a bleeding part edge and proceeds toprocessing at step S180.

At step S180, the CPU 21 determines whether or not the bleeding partedge determination has been performed for all the divided areas 43.Specifically, if i<N×N is satisfied, then the CPU 21 adds 1 to thenumber i identifying the divided area 43 (i=i+1) at step S190 andreturns to step S140 to perform the bleeding part edge determination forthe remaining divided areas. If i=N×N is satisfied, then the CPU 21terminates the processing in FIG. 9.

As described above, in the image processing apparatus 1 of thisembodiment, whether or not a divided area is a bleeding part edge isdetermined with the use of a color edge feature quantity B_(i)calculated based on change of two or more color signals among the Rsignal, the G signal and the B signal, it is possible to relativelydetermine and detect whether or not the divided area is a bleeding partedge. Furthermore, in this embodiment, since it is possible to extractbleeding part edges with various sizes, such as a bleeding part with alarge area and a bleeding part where an edge is extracted being divided,the precision of detecting a bleeding part edge is further enhanced.

Third Embodiment

Next, a third embodiment of the present invention will be described. Inthe first embodiment described above, whether or not a divided area is ableeding part edge is determined with the use of an edge featurequantity on which the amount of change in color signals in a bleedingpart edge is reflected. In this embodiment, an edge feature quantity isalso calculated for peripheral areas, and whether or not a divided areais a bleeding part edge is determined by evaluating continuity of theedge feature quantity from the peripheral areas to a bleeding part edge.

In the image processing apparatus 1 of this embodiment, an imageprocessing program 61, the processing content of which is different fromthat of the image processing program of the image processing apparatus 1in FIG. 2, is used. The image processing apparatus 1 of this embodimentis the same as that of the first embodiment except that analysis data 62acquired, generated and stored in the memory 22 by executing the imageprocessing program 61 includes two kinds of image data, the originalimage 31 and the bleeding part edge image 34 and does not include thebleeding part edge candidate image 32 and the shape edge image 33.Therefore, only characteristic operations will be described here. Thesame components will be given the same reference numerals, anddescription thereof will be omitted.

In this embodiment, processing for detecting, for example, a bleedingpart edge will be described with the use of the flowchart in FIG. 12,similarly to the first embodiment. FIG. 12 is a flowchart illustratingthe procedure for image analysis processing by the image processingprogram 61.

First, at step S201, the CPU 21 acquires image data specified by theoperation device 12 from the image filing apparatus 4 and stores it inthe memory 22 as an original image 31. At the next step S202, the CPU 21generates N×N divided areas 43 from the original image 31 acquired atstep S201 and extracts bleeding part edge candidate areas 44. Since theprocessing at step S202 is similar to the bleeding part edge candidateextraction processing described with the use of FIG. 4 in the firstembodiment, description thereof will be omitted.

At the next step S203, the CPU 21 initializes i which indicates thenumber identifying the bleeding part edge candidate area 44 to beanalyzed, to 1. The number i identifying the bleeding part edgecandidate area 44 is an integer value equal to or above 1 and equal toor below M. Here, M is the number of divided areas 43 extracted as thebleeding part edge candidate areas 44 at step S202.

At the subsequent step S204, the CPU 21 calculates the first to eighth Rchanges in the i-th bleeding part edge candidate area 44. Since themethod for calculating the R changes at step S204 is similar to themethod described with the use of FIG. 7 in the first embodiment,description thereof will be omitted.

At the next step S205, setting the largest value among the first toeighth R changes calculated at step S204 as an edge feature quantityAl_(i), the CPU 21 calculates the direction in which the R change is thelargest, as D_(i). For example, if the fourth R change is the largest,then the direction D_(i) is the diagonally lower-right direction asshown in FIG. 7( d). The two feature quantities, that is, the edgefeature quantity Al_(i) and the direction D_(i) are collectivelyreferred to as a candidate area continuity feature quantity.

Subsequently, at step S206, the CPU 21 sets a background area 63 relatedto the i-th bleeding part edge candidate area 44, as shown in FIG. 13.FIG. 13 is a schematic diagram illustrating the relation between thebleeding part edge candidate area 44 and the background area 63.

The background area 63 is an area required for evaluating the continuityof the edge feature quantity, and it is positioned to be adjacent to thei-th bleeding part edge candidate area 44, in the direction opposite tothe direction D_(i) determined at step S205 when seen from the center ofthe i-th bleeding part edge candidate area 44. The background area 63 isin 1:k similarity relation with the bleeding part edge candidate area44, and, for example, k=2 is set in this embodiment as shown in FIG. 13.

At the next step S207, the CPU 21 calculates the R change in thedirection D_(i) in the background area 63 as a background areacontinuity feature quantity A2 _(i). Since the method for calculatingthe R change is similar to the method for calculating the R changes inthe bleeding part edge candidate area 44, description thereof will beomitted.

At the next step S208, the CPU 21 determines whether or not the i-thbleeding part edge candidate area 44 is a bleeding part edge.Specifically, a value obtained by dividing the background areacontinuity feature quantity A2 _(i) by the edge feature quantity Al_(i)is assumed to be a continuity evaluation feature quantity C_(i), and ifC_(i)≦Th4 is satisfied, then it is determined that the i-th bleedingpart edge candidate area 44 is a bleeding part edge. Here, Th4 is athreshold to be a reference value used when it is determined that ableeding part edge candidate area is a bleeding part edge, and, forexample, Th4=0.2 is set in this embodiment.

As shown in FIG. 14( a), if C_(i)≦Th4 is satisfied, the CPU 21determines at step S209 that the i-th bleeding part edge candidate area44 is a bleeding part edge and proceeds to step S211. As shown in FIG.14( b), if C_(i)>Th4 is satisfied, the CPU 21 determines at step S210that the i-th bleeding part edge candidate area 44 is not a bleedingpart edge but an edge formed by an element other than bleeding andproceeds to processing at step S211.

FIG. 14 is a schematic diagram illustrating bleeding part edgedetermination with the use of the continuity evaluation feature quantityC_(i). FIG. 14 schematically shows the values of R signals of pixelspositioned on the C-C′ line in FIG. 13.

FIGS. 14( a) and 14(b) show change in the value of the R signal in thecase where the bleeding part edge candidate area 44 is a bleeding partedge and change in the value of the R signal in the case where thebleeding part edge candidate area 44 is an edge formed by an elementother than bleeding, respectively.

As shown in FIG. 14( a), in the case where the bleeding part edgecandidate area 44 is a bleeding part edge, the value of the R signal isalmost constant in the background area 63, while the value of the Rsignal largely changes in the bleeding part edge candidate area 44.

On the other hand, as shown in FIG. 14( b), in the case where thebleeding part edge candidate area 44 is an edge formed by an elementother than bleeding, the value of the R signal gradually changes fromthe background area 63 to the bleeding part edge candidate area 44.

By grasping the difference between the changes in the value of the Rsignal from the background area 63 to the bleeding part edge candidatearea 44 as the continuity evaluation feature quantity C_(i), it ispossible to determine whether or not an bleeding part edge candidatearea is a bleeding part edge with the use of the continuity evaluationfeature quantity C_(i).

At step S211, the CPU 21 determines whether or not the edgedetermination has been performed for all the bleeding part edgecandidate areas 44. Specifically, if i<M is satisfied, then the CPU 21adds 1 to the number i identifying the bleeding part edge candidate area44 (i=i+1) at step S212 and returns to step S204 to perform the bleedingpart edge determination for the remaining the bleeding part edgecandidate areas 44. On the other hand, if i=M is satisfied, then the CPU21 terminates the processing in FIG. 12.

As described above, in the image processing apparatus 1 of thisembodiment, it is possible to relatively determine and detect whether ornot a bleeding part edge candidate area is a bleeding part edge by usingan edge feature quantity which is the amount of change in color signalsin a bleeding part edge.

In this embodiment, an edge feature quantity is also calculated forperipheral areas around a bleeding part edge, and whether or not ableeding part edge candidate area is a bleeding part edge is determinedby evaluating the continuity of the edge feature quantity from theperipheral areas to the bleeding part edge. Thereby, it is possible toprevent such shape change as has an edge feature quantity similar tothat of a bleeding part edge or intestinal fluid from being wronglydetected as a bleeding part edge. Thus, according to the presentinvention, the precision of detection of a bleeding part edge is furtherimproved.

Instead of the edge feature quantity, the color edge feature quantitydescribed in the second embodiment may be used to extract bleeding partedge candidate areas 44.

Specifically, the CPU 21 performs processing similar to the bleedingpart edge extraction processing described with the use of FIG. 9 in thesecond embodiment, instead of performing the processing for extractingbleeding part edge candidate areas 44 at step S202 in FIG. 12 (that is,the bleeding part edge candidate extraction processing described withthe use of FIG. 4 in the first embodiment). In this case, the areasextracted as bleeding part edges in the second embodiment are thebleeding part edge candidate areas 44 in this embodiment.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described. Inthe third embodiment described above, an edge feature quantity is alsocalculated for peripheral areas around a bleeding part edge candidatearea, and whether or not a bleeding part edge candidate area is ableeding part edge is determined by evaluating the continuity of theedge feature quantity from the peripheral areas to the bleeding partedge candidate area. In this embodiment, however, whether or not ableeding part edge candidate area is a bleeding part edge is determinedbased on the fluid surface color tone of the internal area of thebleeding part edge candidate area.

The entire configuration of the image processing apparatus 1 is the sameas that in the third embodiment except that the content of processingperformed by an image processing program 71 is different. Therefore,only characteristic operations will be described here. The samecomponents will be given the same reference numerals, and descriptionthereof will be omitted.

In this embodiment, processing for detecting, for example, a bleedingpart edge will be described with the use of the flowchart in FIG. 15,similarly to the third embodiment. FIG. 15 is a flowchart illustratingthe procedure for image analysis processing by an image processingprogram 71.

First, at step S301, the CPU 21 acquires image data specified by theoperation device 12 from the image filing apparatus 4 and stores it inthe memory 22 as an original image 31. At the next step S302, the CPU 21generates N×N divided areas 43 from the original image 31 acquired atstep S301 and extracts bleeding part edge candidate areas 44.

At the next step S303, the CPU 21 initializes i which indicates thenumber identifying the bleeding part edge candidate area 44 to beanalyzed, to 1. The number i identifying the bleeding part edgecandidate area 44 is an integer value equal to or above 1 and equal toor below M. Here, M is the number of the divided areas 43 extracted asthe bleeding part edge candidate areas 44 at step S302.

At the subsequent step S304, the CPU 21 calculates the first to eighth Rchanges in the i-th bleeding part edge candidate area 44. Theprocessings at steps S301 to S304 are similar to the processings atsteps S201 to S204 in FIG. 12, respectively.

At the next step S305, the CPU 21 sets the direction in which the Rchange is the largest among the first to eighth R changes calculated atstep S304, as a candidate area feature quantity D_(i)′. For example, ifthe fourth R change is the largest, then the candidate area featurequantity D_(i)′ is the diagonally lower-right direction as shown in FIG.7( d).

At the subsequent step S306, the CPU 21 sets an internal area 72 relatedto the i-th bleeding part edge candidate area 44 as shown in FIG. 16.FIG. 16 is a schematic diagram illustrating the relation between thebleeding part edge candidate area 44 and the internal area 72.

The internal area 72 is an area for evaluating the fluid surface colortone inside an edge, and it is positioned to be adjacent to the i-thbleeding part edge candidate area 44 in the direction of the candidatearea feature quantity D_(i)′ determined at step S305 when seen from thecenter of the i-th bleeding part edge candidate area 44. Furthermore,the internal area 72 has the same shape and area as the bleeding partedge candidate area 44.

Next, at step S307, the CPU 21 acquires the values of R signals, Gsignals and B signals of the pixels included in the internal area 72 andcalculates average values Ra_(i)′, Ga_(i)′ and Ba_(i)′ of the respectivecolor signals in the internal area 72. At the subsequent step S308, theCPU 21 compares a bleeding part area color tone space set in advance andthe average values Ra_(i)′, Ga_(i)′ and Ba_(i)′ of the respective colorsignals calculated at step S307.

The processings at steps S307 and S308 are similar to the processings atsteps S13 and S14 in FIG. 3, respectively. If the average valuesRa_(i)′, Ga_(i)′ and Ba_(i)′ of the respective color signals calculatedat step S307 exist within the bleeding part area color tone space, theCPU 21 determines at step S309 that the i-th bleeding part edgecandidate area 44 is a bleeding part edge and proceeds to processing atstep S311.

On the other hand, if the average values Ra_(i)′, Ga_(i)′ and Ba_(i)′ ofthe respective color signals calculated at step S307 do not exist withinthe bleeding part area color tone space, then the CPU 21 determines atstep S310 that the i-th bleeding part edge candidate area 44 is not ableeding part edge and proceeds to processing at step S311.

The G/R and B/G color tone space may be used to make determination on ableeding part edge. That is, if (G/R)min≦Ga_(i)/Ra_(i)≦(G/R)max and(B/G)min≦Ba_(i)/Ga_(i)≦(B/G)max are satisfied, then the CPU 21determines that the i-th divided area 43 is a bleeding part edgecandidate.

At step S311, the CPU 21 determines whether or not the edgedetermination has been performed for all the bleeding part edgecandidate areas 44. Specifically, if i<M is satisfied, then the CPU 21adds 1 to the number i identifying the bleeding part edge candidate area44 at step S311 and returns to step S304 to perform the bleeding partedge determination for the remaining bleeding part edge candidate areas44. On the other hand, if i=M is satisfied, then the CPU 21 terminatesthe processing in FIG. 15.

As described above, in the image processing apparatus 1 of thisembodiment, it is possible to relatively determine and detect whether ornot a bleeding part edge candidate area is a bleeding part edge by usingan edge feature quantity which is the amount of change in color signalsin a bleeding part edge.

In this embodiment, whether or not a bleeding part edge candidate areais a bleeding part edge is determined by evaluating the fluid surfacecolor tone in the internal area of a bleeding part edge. Thereby, it ispossible to prevent such a foreign matter as has a change of an edgefeature quantity similar to that of a bleeding part edge, from theperipheral mucous membrane to a bleeding part or intestinal fluid frombeing wrongly detected as a bleeding part edge. Thus, according to thepresent embodiment, the precision of detection of a bleeding part edgeis further improved.

In this embodiment also, the color edge feature quantity described inthe second embodiment may be used to extract bleeding part edgecandidate areas 44, instead of the edge feature quantity, similarly tothe third embodiment.

In the above four embodiments, the case of extracting a bleeding partedge given has been described as an example. However, the presentinvention is not limited to the embodiments described above, and variouschanges and alterations can be made within the range where the spirit ofthe present invention is not changed. For example, the present inventionis applicable to the case of extracting the outline part of a reddenedpart on the surface of mucous membrane.

Fifth Embodiment

Next, a fifth embodiment of the present invention will be describedbelow with reference to FIG. 1 and FIGS. 17 to 25.

The object of this embodiment is to provide an image processingapparatus, an image processing method and an image processing programcapable of relatively determining whether or not an area is a hemorrhageedge with the use of the amount of change in color signals on theoutline part of a hemorrhage area (hereinafter referred to as ahemorrhage edge) and detecting a hemorrhage area surrounded by thehemorrhage edge.

As examples of the hemorrhage area, a bleeding part where bleeding frommucous membrane actually occurs, a reddened part where the surface ofmucous membrane is reddened by hyperemia, and the like are given, asdescribed in the first embodiment.

In this embodiment, the case of detecting, for example, a bleeding partwill be described. The network configuration formed by the imageprocessing apparatus 1 according to the fifth embodiment of the presentinvention and its related systems is the same configuration in FIG. 1.

As shown in FIG. 1, the image processing apparatus 1 which performsvarious image processings and information processings is connected tothe LAN 2 which uses TCP/IP as its communication protocol. Meanwhile,the endoscope observation apparatus 3 which captures an image of theinside of a living body and outputs an image signal is also connected tothe LAN 2 via the endoscope filing apparatus 4.

The endoscope filing apparatus 4 receives an image signal from theendoscope observation apparatus 3, generates image data and accumulatesthe generated image data. That is, the image processing apparatus 1acquires the image data accumulated in the endoscope filing apparatus 4via the LAN 2.

The image processing apparatus 1 of this embodiment is configured asshown in FIG. 17. The hardware configuration of this image processingapparatus 1 is the same as that of the image processing apparatus 1 inFIG. 2. Therefore, the same components as in FIG. 2 are given the samereference numerals. In the image processing apparatus 1 of thisembodiment, an image processing program 128 different from the imageprocessing program 28 in FIG. 2 is adopted. Furthermore, in thisembodiment, images different from the analysis data 29 stored in thememory 22 in FIG. 2 are stored as the analysis data 29 stored in thememory 22.

As shown in FIG. 17, the image processing apparatus 1 of this embodimentis configured with the general-purpose personal computer 11 as itscenter and provided with the operation device 12 configured by akeyboard and a mouse, the storage device 13 configured by a hard diskand the display device 14 configured by a CRT.

Each of the operation device 12, the storage device 13 and the displaydevice 14 is electrically connected to the personal computer 11.Specification of image data to be processed, acquisition and display ofthe specified image data and instruction to execute processing areinputted from the operation device 12. The result of various processingsperformed by the image processing apparatus 1 is displayed on thedisplay device 14.

The personal computer 11 has the CPU 21 which performs execution andcontrol of various programs, the memory 22 which stores the variousprocessing programs and data, the external storage I/F 23 which performsreading and writing of information to and from the storage device 13,the network card 24 which communicates information with externalequipment, the operation I/F 25 which receives an operation signalinputted from the operation device 12 and performs necessary dataprocessing and the graphic board 26 which outputs a video signal to thedisplay device 14, and each of them is electrically connected to the bus27. Therefore, these components in the personal computer 11 can mutuallysend and receive information via the bus 27.

The network card 24 is electrically connected to the LAN 2 and sends andreceives information to and from the endoscope filing apparatus 4 whichis also connected to the LAN 2.

The external storage I/F 23 reads an image processing program 128 storedin the storage device 13 and stores it in the memory 22. The imageprocessing program 128 is a program which executes image analysisprocessing and configured by multiple execution files, dynamic linklibrary files or setting files.

By the image processing program 128 which is stored in the memory 22being executed, the CPU 21 operates. The CPU 21 performs image analysisprocessing for image data acquired from the endoscope filing apparatus4. Analysis data 29 acquired or generated by each processing by the CPU21 is stored in the memory 22.

This analysis data 29 includes an original image 31 which is the imagedata acquired from the endoscope filing apparatus 4. Furthermore, theanalysis data 29 includes an edge image 132 generated by variousprocessings to be described later, a bleeding part candidate image 133and a bleeding part image 134.

In this case, the CPU 21 has an evaluation area setting processingfunction of dividing the original image 31 into multiple small areas,extracting areas including the outline part of a hemorrhage area basedon color signals in each of the small areas, and setting the extractedsmall areas as hemorrhage evaluation areas.

When performing the evaluation area setting processing, the CPU 21 setsmultiple small areas in the vicinity of or around the hemorrhageevaluation area as evaluation target areas. The CPU 21 has a hemorrhagecandidate area determination processing function of calculating theamount of change of color signals in the multiple evaluation targetareas, extracting hemorrhage edge candidate areas based on the amount ofchange, and determining whether or not the hemorrhage evaluation area isa hemorrhage candidate area based on the ratio of the set evaluationtarget areas to the extracted hemorrhage edge candidate areas.

This CPU 21 also has a hemorrhage area determination processing functionof determining whether or not a hemorrhage candidate area is ahemorrhage area based on the amount of change of two or more colorsignals in the hemorrhage edge candidate area.

The operation of the image processing apparatus 1 configured asdescribed above will be described. In this embodiment, the case ofdetecting, for example, a bleeding part area will be described with theuse of the flowchart in FIG. 18.

FIG. 18 is a flowchart illustrating the procedure for image analysisprocessing by an image processing program 128. First, at step S401,which is an original image acquisition step, the CPU 21 acquires imagedata specified by the operation device 12 from the image filingapparatus 4 and stores it in the memory 22 as an original image 31.

The original image 31 is a color image constituted by three planes ofred (R), green (G) and blue (B). The tone number of a pixel of eachplane takes an 8-bit value, that is, a value between 0 and 255. Next, atstep S402, which is an image analysis processing step, the CPU 21performs various processings for the original image 31 acquired at stepS401 to generate a bleeding part candidate image 133, an edge image 132and a bleeding part image 134.

This image analysis processing step (step S402) is constituted by edgeextraction processing (step S410) for generating the edge image 132 fromthe original image 31, bleeding part candidate extraction processing(step S420) for generating the bleeding part candidate image 133 fromthe original image 31 and bleeding part determination processing (stepS440) for generating the bleeding part image 134 from the edge image 132and the bleeding part candidate image 133, and the processings areexecuted in that order. Each of the above processings in the imageanalysis processing step (step S402) will be described in accordancewith the processing order.

First, the edge extraction processing will be described with the use ofFIG. 19. FIG. 19 is a flowchart illustrating the procedure for the edgeextraction processing.

First, at step S411, the CPU 21 divides the original image 31 into N×Nsmall areas. In the embodiment of the present invention, for example,N=36 is set. Reverse gamma correction or shading correction for theoriginal image 31 may be added as preprocessing before step S411. Inthis case, the corrected image for which the correction processing hasbeen performed is divided into small areas, and subsequent processingsare performed therefor.

Next, at step S412, the CPU 21 initializes i which indicates the numberidentifying the divided small area (hereinafter referred to as a dividedarea) to be analyzed, to 1. The number identifying the divided area 141takes an integer value equal to or above 1 and equal to or below N×N.

Next, at step S413, the CPU 21 calculates change in the value of the Gsignal (hereinafter referred to as G change) in the i-th divided area141. The G change is calculated from the value of the G signal (G1) of aparticular pixel in the divided area 141 and the value of the G signal(G2) of a different, particular pixel in the same divided area 43.Specifically, the CPU 21 performs calculation with the formula of Gchange=log_(e)(G2)−log_(e)(G1).

In this embodiment, the CPU 21 calculates the G change for each of eightdirections, that is, upward and downward directions, right and leftdirections and diagonal directions in the divided area 141, as shown inFIG. 20. FIGS. 20( a) to 20(h) are diagrams illustrating a method forcalculating the G changes.

First G change is upward G change as shown in FIG. 20( a), and it iscalculated with the value of the G signal of the pixel at the lowercenter as G1 and the value of the G signal of the pixel at the uppercenter as G2. Second G change is G change toward the diagonallyupper-right direction as shown in FIG. 20( b), and it is calculated withthe value of the G signal of the pixel at the lower left as G1 and thevalue of the G signal of the pixel at the upper right as G2. Third Gchange is the right-direction G change as shown in FIG. 20( c), and itis calculated with the value of the G signal of the pixel at the leftcenter as G1 and the value of the G signal of the pixel at the rightcenter as G2.

Fourth G change is G change toward the diagonally lower-right directionas shown in FIG. 20( d), and it is calculated with the value of the Gsignal of the pixel at the upper left as G1 and the value of the Gsignal of the pixel at the lower right as G2. Fifth G change is thedownward G change as shown in FIG. 20( e), and it is calculated with thevalue of the G signal of the pixel at the upper center as G1 and thevalue of the G signal of the pixel at the lower center as G2. Sixth Gchange is the G change toward the diagonally lower-left direction asshown in FIG. 20( f), and it is calculated with the value of the Gsignal of the pixel at the upper right as G1 and the value of the Gsignal of the pixel at the lower left as G2.

Seventh G change is G change toward the left direction as shown in FIG.20( g), and it is calculated with the value of the G signal of the pixelat the right center as G1 and the value of the G signal of the pixel atthe left center as G2. Eighth G change is G change toward the diagonallyupper-left direction as shown in FIG. 20( h), and it is calculated withthe value of the G signal of the pixel at the lower right as G1 and thevalue of the G signal of the pixel at the upper left as G2.

Next, at step S414, setting the largest value among the first to eighthG changes calculated at step S413 as Eli, the CPU 21 calculates thedirection in which the G change is the largest, as D1 _(i). For example,if the fourth G change is the largest, then the direction D1 _(i) is thediagonally lower-right direction as shown in FIG. 20( d). The twofeature quantities, that is, the largest value of the G change Eli andthe direction D1 _(i) are collectively referred to as an edge featurequantity.

Next, at step S415, the CPU 21 determines whether or not the i-thdivided area 141 is an edge. Specifically, if the largest value of the Gchange E1 _(i)>Thr1 is satisfied, then the CPU 21 determines that thei-th divided area 141 is an edge. Here, Thr1 is a threshold, and, forexample, Thr1=0.3 is set in the embodiment of the present invention.

If E1 _(i)>Thr1 is satisfied, then the CPU 21 determines at step S416that the i-th divided area 141 is an edge and proceeds to step S418. IfE1 _(i)≦Thr1 is satisfied, then the CPU 21 determines at step S417 thatthe i-th divided area 141 is not an edge and proceeds to step S418.

At step S418, the CPU 21 determines whether or not the edgedetermination has been performed for all the divided areas 141.Specifically, if i<N×N is satisfied, then the CPU 21 adds 1 to thenumber i identifying the divided area 141 (i=i+1) at step S419 andreturns to step S413 to perform the edge determination for the remainingdivided areas 141. If i=N×N is satisfied, then the CPU 21 terminates theprocessing in FIG. 19 and proceeds to the subsequent bleeding partcandidate extraction processing.

Next, the bleeding part candidate extraction processing will bedescribed with the use of FIG. 21. FIG. 21 is a flowchart illustratingthe procedure for the bleeding part candidate extraction processing. Inthe bleeding part candidate extraction processing, the CPU 21 extractsbleeding part candidates and bleeding part edge candidates based on thearrangement of the edge extracted by the edge extraction processing.

First, the CPU 21 executes the processings at steps S421 to S423 asevaluation area setting means, and sets peripheral areas 143 asevaluation target areas, for the i-th divided area 141 as a hemorrhageevaluation area.

Here, the peripheral areas 143 are set on a closed curve surrounding theperiphery of the i-th divided area 141. At step S421, the CPU 21initializes i which indicates the number identifying the divided area141 to be analyzed, to 1. The number i identifying the divided area 141takes an integer value equal to or above 1 and equal to or below N×N.

Next, at step S422 as shown in FIG. 22( a), the CPU 21 acquires M×Mdivided areas 141 with the i-th divided area 141 as the center, asarrangement evaluation areas 142 for the i-th divided area 141.

FIG. 22 is a diagram illustrating the bleeding part candidate extractionprocessing. In this embodiment, the CPU 21 sets M=5, for example.Subsequently, at step S423, the CPU 21 acquires divided areas 141 whichare in contact with the outline of the arrangement evaluation areas 142acquired at step S422, as the peripheral areas 143 (shaded areas).

In this embodiment, the number C1 _(i) of the peripheral areas 143related to the i-th divided area 141 is sixteen as shown in FIG. 22( a).

Next, the CPU 21 executes processings at steps S424 to S432 ashemorrhage candidate area determination means. First, at step S424, theCPU 21 initializes j which indicates the number identifying theperipheral area 143 to be analyzed and a counter Cnt1 for counting thenumber of bleeding part edge candidate areas among the peripheral areas143, to 1 and 0, respectively.

The number j which identifies the peripheral area 143 and the counterCnt1 take an integer value equal to or above 1 and equal to or below C1_(i). Subsequently, at step S425, the CPU 21 sets the direction fromj-th peripheral area 143 toward the i-th divided area 141 as D2 _(ij).Next, at step S426, the CPU 21 calculates G changes in eight directionsin the j-th peripheral area 43 and sets the direction in which the Gchange is the largest as D3 _(ij). The method for calculating the Gchanges at step S426 is similar to the method for calculating the Gchanges at step S413.

Subsequently, at step S427, the CPU 21 determines an angle θ_(ij) formedby the direction D2 _(ij) acquired at step S425 and the direction D3_(ij) acquired at step S426 (see FIG. 22( b)). Next, at step S428, theCPU 21 determines whether or not the j-th peripheral area 143 is ableeding part edge candidate. Specifically, for the angle θ_(ij)calculated at step S427, the CPU 21 determines whether or notθ_(ij)≦Thr2 is satisfied. Here, Thr2 is a threshold, and, for example,Thr2=45 (degrees) is set in the embodiment of the present invention. Thebleeding part edge candidate determination at step S428 is tentative(interim) determination. The result of the determination at this step isnot adopted until it is determined by the processing to be describedlater that the i-th divided area 141 is a bleeding part candidate area.

If θ_(ij)≦Thr2 is satisfied, then the CPU 21 determines that the j-thperipheral area 143 is a bleeding part edge candidate, adds 1 to Cnt1 atstep S429 and proceeds to step S430. If of θ_(ij)>Thr2 is satisfied,then the CPU 21 determines at step S428 that the j-th peripheral area143 is not a bleeding part edge candidate and proceeds to step S430.

At step S430, the CPU 21 determines whether or not the bleeding partedge candidate determination has been performed for all the peripheralareas 143 related to the i-th divided area 141. Specifically, if j<C1_(i) is satisfied, then the CPU 21 adds 1 to the number j identifyingthe peripheral area 143 (j=j+1) at step S431 and returns to step S425 toperform the bleeding part edge candidate determination for the remainingperipheral areas 143. If j=C1 _(i) is satisfied, then the CPU 21proceeds to step S432.

At step S432, the CPU 21 determines whether or not the i-th divided area41 is a bleeding part candidate. Specifically, the ratio of Cnt1 whichis the number of the bleeding part edge candidate areas to C1 _(i) whichis the number of the peripheral areas 143 is calculated, and it isdetermined whether Cnt1/Cl_(i)>Thr3 is satisfied. Here, Thr3 is athreshold, and, for example, Thr3=0.7 is set in the embodiment of thepresent invention.

As shown in FIG. 23, if Cnt1/C1 _(i)>Thr3 is satisfied, then the CPU 21determines at step S433 that the i-th divided area 141 is a bleedingpart candidate. At step S428, the CPU 21 formally determines that theperipheral areas 143 tentatively determined to be bleeding part edgecandidates, that is, peripheral areas 143 which are related to the i-thdivided area 141 and satisfy θ≦Thr2 are bleeding part edge candidates.FIG. 23 is a diagram illustrating a method for determining bleeding partcandidates.

In FIG. 23, for example, among sixteen peripheral areas 143 related tothe i-th divided area 141, fourteen peripheral areas 143 where thedirection D1 _(i), which is the edge feature quantity, is shown as anarrow are determined to be edges. Furthermore, since twelve among thefourteen peripheral areas determined to be edges satisfy θ_(ij)≦Thr2,the result of Cnt1/C1 _(i) is 0.75.

Since the value of Cnt1/C1 _(i) is above 0.7, which is set as Thr3, thei-th divided area 141 is determined to be a bleeding part candidate, andthe twelve peripheral areas 143 satisfying θ_(ij)≦Thr2 are determined tobe bleeding part edge candidates. (The meshed peripheral areas 143 inFIG. 23 are the areas determined to be bleeding part edge candidates.)

When step S433 ends, the CPU 21 proceeds to step S434. If Cnt1/C1_(i)≦Thr3 is satisfied, then the CPU 21 determines that the i-th dividedarea 141 is not a bleeding part candidate and proceeds to step S434.

At step S434, the CPU 21 determines whether or not the bleeding partcandidate determination has been performed for all the divided areas141. Specifically, if i<N×N is satisfied, then the CPU 21 adds 1 to thenumber i identifying the divided area 141 (i=i+1) at step S435 andreturns to step S422 to perform the bleeding part candidatedetermination for the remaining divided areas 141. If i=N×N issatisfied, the CPU 21 terminates the processing in FIG. 21 and proceedsto the subsequent bleeding part determination processing.

Next, the bleeding part determination processing as hemorrhage areadetermination means will be described with the use of FIG. 24. FIG. 24is a flowchart illustrating the procedure for the bleeding partdetermination processing. In the bleeding part determination processing,for each of the peripheral areas 143 around the bleeding part edgecandidate extracted by the bleeding part candidate extractionprocessing, the CPU 21 calculates a color edge feature quantity based onthe ratio of changes of two or more color signals among the R signal,the G signal and the B signal, and determines whether or not theperipheral area is a bleeding part edge based on the color edge featurequantity. Furthermore, the CPU 21 determines a bleeding part area basedon the extracted bleeding part edge areas.

First, at step S441, the CPU 21 acquires the divided areas 141determined to be bleeding part candidates by the bleeding part candidateextraction processing. The number of the divided areas 141 which arebleeding part candidates is denoted by H. Next, at step S442, the CPU 21initializes k which indicates the number identifying the divided area141 to be analyzed, to 1. The divided areas 141 to be analyzed are the Hdivided areas 141 which are the bleeding part candidates acquired atstep S441. Therefore, the number k identifying the divided area 141takes an integer value equal to or above 1 and equal to or below H.

Subsequently, at step S443, the CPU 21 acquires the peripheral areas 143determined to be bleeding part edge candidates around the k-th dividedarea 141 in the bleeding part candidate extraction processing. Thenumber of the peripheral areas 143 which are bleeding part edgecandidates is denoted by C2 _(k). Next, at step S444, the CPU 21initializes 1 which indicates the number identifying the peripheral area143 to be analyzed and a counter Cnt2 for counting the number ofbleeding part edge areas among the peripheral areas 143, to 1 and 0,respectively. The number 1 which identifies the peripheral area 143 andthe counter Cnt2 take an integer value equal to or above 1 and equal toor below C2 _(k).

Next, at step S445, the CPU 21 calculates change in the value of the Rsignal (hereinafter referred to as R change) and change in the value ofthe G signal (hereinafter referred to as G change) in the l-thperipheral area 143. The R change is calculated from the value of the Rsignal (R1) of a particular pixel P1 in the peripheral area 143 and thevalue of the R signal (R2) of a different particular pixel P2 in thesame peripheral area 143. Specifically, it is calculated with theformula of R change=log_(e)(R2)−log_(e)(R1).

The G change is calculated from the value of the G signal of the pixelP1 (G1) and the value of the G signal of the pixel P2 (G2) which areused when the R change is calculated. Specifically, it is calculatedwith the formula of G change=log_(e)(G2)−log_(e)(G1). In thisembodiment, the R change and the G change are calculated for each ofeight directions, that is, upward and downward directions, right andleft directions and diagonal directions in the peripheral area 143, asshown in FIG. 25.

FIGS. 25( a) to 25(h) are diagrams illustrating a method for calculatingR changes and G changes. Since the method for calculating the first toeighth G changes shown in FIGS. 25( a) to 25(h) is similar to the methodfor calculating the first to eighth G changes shown in FIGS. 20( a) to20(h) calculated at step S413, detailed description thereof will beomitted.

In the method for calculating the first to eighth R changes shown inFIGS. 25( a) to 25(h), the same pixels used in calculation of the firstto eighth G changes are used, and the method is similar to the methodfor calculating the first to eighth G changes if G1 and G2, the valuesof the G signal, are substituted with R1 and R2, the values of the Rsignal, respectively. Therefore, detailed description thereof will beomitted.

Next, at step S446, the CPU 21 determines the first to eighth changeratios by dividing the first to eighth G changes by the first to eighthR changes, respectively. Furthermore, the CPU 21 sets the value of theratio of the G change to the R change in the direction D3 _(kl) in whichthe G change becomes the largest in the l-th peripheral area 143 andwhich has been calculated at step S426, as a color edge feature quantityB11.

In the vicinity of a boundary area between peripheral mucous membraneand a bleeding part, that is, in the vicinity of a bleeding part edge,change in the G signal is generally larger than change in the R signalor the B signal. Therefore, the largest value of G change/R change isassumed to be the color edge feature quantity. The largest value of Gchange/B change may be used as the color edge feature quantity.

Next, at step S447, the CPU 21 determines whether or not the l-thperipheral area 143 is a bleeding part edge. Specifically, if the coloredge feature quantity B1 _(l)>Thr4 is satisfied, then the CPU 21determines that the l-th peripheral area 143 is a bleeding part edge.The bleeding part edge determination at step S447 is tentativedetermination. The result of the determination at this step is notadopted until it is determined by the processing to be described laterthat the k-th divided area 141 is a bleeding part area.

Here, Thr4 is a threshold, and, for example, Thr4=1.0 is set in theembodiment of the present invention. If B1 _(l)>Thr4 is satisfied, thenthe CPU 21 determines that the l-th peripheral area 143 is a bleedingpart edge, adds 1 to Cnt2 at step S448 and proceeds to step S449.

If B1 _(l)≦Thr4 is satisfied, then the CPU 21 determines that the l-thperipheral area 143 is not a bleeding part edge and proceeds to stepS449.

At step S449, the CPU 21 determines whether or not the bleeding partedge determination has been performed for all the peripheral areas 143related to the k-th divided area 141. Specifically, if l<C2 _(k) issatisfied, then the CPU 21 adds 1 to the number l identifying theperipheral area 143 (l=l+1) at step S450 and returns to step S445 toperform the bleeding part edge determination for the remainingperipheral areas 143.

If j=C2 _(k) is satisfied, then the CPU 21 terminates the bleeding partedge determination processing and proceeds to step S451.

At step S451, the CPU 21 determines whether or not the k-th divided area141 is a bleeding part. Specifically, the CPU 21 calculates the ratio ofCnt2 which is the number of the bleeding part edge areas to C2 _(k)which is the number of the peripheral areas 143 and determines whetherCnt2/C2 _(k)>Thr5 is satisfied.

Here, Thr5 is a threshold, and, for example, Thr5=0.7 is set in theembodiment of the present invention. If Cnt2/C2 _(k)>Thr5 is satisfied,then the CPU 21 determines at step S452 that the k-th divided area 141is a bleeding part.

At step S447, the CPU 21 formally determines that the peripheral areas143 tentatively determined to be bleeding part edges, that is,peripheral areas 143 which are related to the k-th divided area 141 andsatisfy B1 _(l)>Th4 are bleeding part edges.

When step S452 ends, the CPU 21 proceeds to step S453. If Cnt2/C2_(k)<Thr5 is satisfied, then the CPU 21 determines that the k-th dividedarea 141 is not a bleeding part and proceeds to step S453.

At step S453, the CPU 21 determines whether or not the bleeding partdetermination has been performed for all the divided areas 141 that arebleeding part candidates. Specifically, if k<H is satisfied, then theCPU 21 adds 1 to the number k identifying the divided area 141 at stepS454 (k=k+1) and returns to step S443 to perform the bleeding partdetermination for the remaining divided areas 141 which are bleedingpart candidates. If k=H is satisfied, then the CPU 21 terminates theprocessing.

As described above, in the image processing apparatus 1 of thisembodiment, bleeding part candidates are extracted based on thearrangement of bleeding part edge candidates; it is relativelydetermined whether or not a bleeding part edge candidate is a bleedingpart edge based on the ratio of the amount of change in different colorsignals (color edge feature quantity) in the bleeding part edgecandidate; and an area surrounded by the bleeding part edge is detectedas a bleeding part. Thereby, a bleeding part with a small area can bedetected with a high precision.

Sixth Embodiment

Next, a sixth embodiment of the present invention will be described. Inthe fifth embodiment described above, divided areas 141 which are incontact with the outline of arrangement evaluation areas 142 are assumedto be peripheral areas 143. In this embodiment, peripheral areas 143 arealso provided within the arrangement evaluation areas 142 so that therange of the hemorrhage edge candidate evaluation target area iswidened. Thereby, it is possible to detect bleeding parts in morevarious shapes.

The entire configuration of the image processing apparatus 1 is the sameas that of the fifth embodiment except that the content of theprocessing performed by an image processing program 81 is different fromthat of the image processing program 128 in the fifth embodiment.Therefore, only characteristic operations will be described here. Thesame components will be given the same reference numerals, anddescription thereof will be omitted. In this embodiment, the case ofdetecting, for example, a bleeding part will be described, similarly tothe fifth embodiment.

The image analysis processing in this embodiment is the same as that ofthe fifth embodiment except that steps S423 to S431 of the bleeding partcandidate extraction processing, that is, the content of the bleedingpart edge candidate determination processing is different. The bleedingpart edge candidate determination processing in this embodiment will bedescribed with the use of the flowchart in FIG. 26.

FIG. 26 is a flowchart illustrating the procedure for the bleeding partedge candidate determination processing in the sixth embodiment.

At step S401 in FIG. 18, the CPU 21 acquires an original image 31, andacquires arrangement evaluation areas 142 for the i-th divided area 141through steps S421 to S422 in FIG. 21 after finishing the edgeextraction processing at step S410. Then, the bleeding part edgecandidate determination processing shown in FIG. 26 is executed.

In the bleeding part edge candidate determination processing, first atstep S523, the CPU 21 sets the number of divided areas 141 within zareas from the outline part as C3 _(i) which is the number of theperipheral areas 143, within the arrangement evaluation areas 142acquired at step S22. In this embodiment, for example, divided areas 141corresponding to 5×5=25 are acquired as the arrangement evaluation areas142, and as for Z corresponding to the width of the peripheral areas143, Z=2 is set. In this case, C3 _(i), the number of the peripheralareas 143 is eight, as shown in FIG. 27. FIG. 12 is a diagramillustrating the positions where the peripheral areas 143 are set in thesecond embodiment.

Next, at step S524, the CPU 21 initializes m which indicates the numberidentifying the peripheral area 143 to be analyzed and a counter Cnt3for counting the number of bleeding part edge candidate areas in theperipheral areas 143, to 1 and 0, respectively.

The number m which identifies the peripheral area 143 and the counterCnt3 take an integer value equal to or above 1 and equal to or below C3_(i). Subsequently, at step S525, the CPU 21 acquires divided areas 141included in the m-th peripheral area 143.

Specifically, half lines are drawn from the i-th divided area 141positioned in the center of the arrangement evaluation areas 142 to theoutline of the arrangement evaluation areas 142 through Z divided areas141 positioned inside the outline of the arrangement evaluation areas142, and divided areas 141 positioned on the half lines are acquired asthe peripheral areas 143 (see FIG. 27).

In this embodiment, since Z=2 is set, the number of divided areas 141included in one peripheral area 143 is two. That is, in FIG. 27, twodivided areas 141 connected via a line segment belong to the sameperipheral area 143.

Subsequently, at step S526, the CPU 21 initializes n which indicates thenumber identifying the divided areas 141 in the m-th peripheral area143, to 1. The number n identifying the divided area 141 takes aninteger value equal to or above 1 and equal to or below Z.

Next, at step S527, the CPU 21 sets the direction from n-th divided area141 in the m-th peripheral area 143 to the i-th divided area 141 as D4_(imn). Subsequently, at step S528, the CPU 21 determines an angleθ_(imn) formed by the direction D1 _(imn) of the edge feature quantityof the n-th divided area 141 in the m-th peripheral area 143, which hasbeen acquired at step S14, and the direction D4 _(imn) acquired at stepS527.

Subsequently, at step S529, the CPU 21 determines whether or not them-th peripheral area 143 is a bleeding part edge candidate.Specifically, as for the angle θ_(imn) calculated at step S528, the CPU21 determines whether or not θ_(imn)≦Thr6 is satisfied.

Here, Thr6 is a threshold, and, for example, Thr6=45 (degrees) is set inthe embodiment of the present invention. Similarly to step S428 in thefifth embodiment, the bleeding part edge candidate determination at stepS529 is tentative determination. The result of the determination at thisstep is not adopted until it is determined by the processing to bedescribed later that the i-th divided area 141 is a bleeding partcandidate area. If θ_(imn)≦Thr6 is satisfied, then the CPU 21 determinesthat the m-th peripheral area 143 is a bleeding part edge candidate,adds 1 to Cnt3 at step S530 and proceeds to step S533.

If θ_(imn)>Thr6 is satisfied, then at step S531, the CPU 21 determineswhether or not the bleeding part edge candidate determination has beenperformed for all the divided areas 141 included in the m-th peripheralarea 143. Specifically, if n<Z is satisfied, then the CPU 21 adds 1 tothe number n identifying the divided area 141 in the m-th peripheralarea 143 (n=n+1) at step S532 and returns to step S527 to perform thebleeding part edge candidate determination for the remaining dividedareas 141.

If n=Z is satisfied, then the CPU 21 determines that the m-th peripheralarea 143 is not a bleeding part edge candidate and proceeds to stepS533. That is, if there is any area that satisfies θ_(imn)≦Thr6, whichis the condition for determination of a bleeding part edge candidate,among the Z divided areas 141 included in the m-th peripheral area, thenthe CPU 21 determines that the m-th peripheral area is a bleeding partedge candidate.

At step S533, the CPU 21 determines whether or not the bleeding partedge candidate determination has been performed for all the peripheralareas 143 related to the i-th divided area 141. Specifically, if m<C3_(i) is satisfied, then the CPU 21 adds 1 to the number m identifyingthe peripheral area 143 (m=m+1) at step S534 and returns to step S525 toperform the bleeding part edge candidate determination for the remainingperipheral areas 143.

If m=C3 _(i) is satisfied, then the CPU 21 terminates the processing andproceeds to step S32. Since the processings after step S32 are similarto those in the fifth embodiment, description thereof will be omitted.

When the bleeding part candidate determination processing ends at stepS34, bleeding part candidates are extracted as shown in FIG. 28, forexample. FIG. 28 is a diagram illustrating the bleeding part candidatedetermination processing. In FIG. 28, for example, it is determinedthat, among eight peripheral areas 143 related to the i-th divided area141, eight peripheral areas 143 in which the direction D1 _(imn) whichis the edge feature quantity, is shown as an arrow in a divided area 141are edges.

Furthermore, since all the eight peripheral areas 143 which have beendetermined to be edges satisfy θ_(imn)≦Thr6, the result of Cnt3/C3 _(i)becomes 1. Since the value of Cnt3/C3 _(i) is above 0.7 set as Thr3, thei-th divided area 141 is determined to be a bleeding part candidate, andthe eight peripheral areas 143 satisfying θ_(imn)≦Thr6 are determined tobe bleeding part edge candidates.

In the bleeding part determination processing for the bleeding partcandidates extracted as described above, bleeding part determination isperformed based on the color edge feature quantity of the bleeding partedge candidates, similarly to the fifth embodiment.

As described above, in the image processing apparatus 1 of thisembodiment, since the range of the peripheral areas 143 to be evaluatedas a bleeding part edge candidate is widened, it is possible to detect ableeding part in various forms such as an oval and an amoeba shape inaddition to small bleeding parts, and the precision of detecting ableeding part is further improved.

Seventh Embodiment

Next, a seventh embodiment of the present invention will be described.In the fifth embodiment described above, divided areas 141 in contactwith the entire circumference of the outline of arrangement evaluationareas 142 are assumed to be peripheral areas 143. In this embodiment,divided areas 141 arranged on opposed two sides of the outline of thearrangement evaluation areas 142 and on opposed two lines with adiagonal line between them are assumed to be peripheral areas 143.

Thereby, even when a bleeding part is belt-shaped and only a part of thebleeding part is shown in an endoscope image, it is possible to detectthe bleeding part according to this embodiment.

The entire configuration of the image processing apparatus 1 is the sameas that of the fifth embodiment except that the content of theprocessing performed by an image processing program 91 is different fromthat of the image processing program 128 in the fifth embodiment.Therefore, only characteristic operations will be described here. Thesame components will be given the same reference numerals, anddescription thereof will be omitted. In this embodiment, the case ofdetecting, for example, a bleeding part will be described, similarly tothe fifth embodiment.

The image analysis processing in this embodiment is the same as that ofthe fifth embodiment except that the positions of peripheral areas 143set at step S423 in the bleeding part candidate extraction processingare different, and that there are provided multiple patterns ofarrangement of peripheral areas 143 set for one divided area 141 asshown in FIGS. 29( a) to 29(d).

That is, at step S423, the CPU 21 acquires divided areas 141 which arein contact with vertically opposed two sides of the outline of thearrangement evaluation areas 142 acquired at step S422, as peripheralareas 143, as shown in FIG. 29( a), and performs the bleeding part edgecandidate determination and bleeding part candidate determinationprocessings through steps S424 to S433.

When step S433 ends, the CPU 21 returns to step S423, and it acquiresdivided areas 141 which are in contact with horizontally opposed twosides of the outline of the arrangement evaluation areas 142, asperipheral areas 143, as shown in FIG. 29( b), and performs the bleedingpart edge candidate determination and bleeding part candidatedetermination processings.

Similarly, the CPU 21 performs the bleeding part edge candidatedetermination and bleeding part candidate determination processings forthe pattern in which divided areas 141 positioned on two lines with adiagonal line from the upper left toward the lower right between themare set as peripheral areas 143 as shown in FIG. 29( c), and the patternin which divided areas 141 positioned on two lines with a diagonal linefrom the upper right toward the lower left between them are set asperipheral areas 143 as shown in FIG. 29( d).

After performing the bleeding part edge candidate determination andbleeding part candidate determination processings for all thearrangement patterns for peripheral areas 143, the CPU 21 proceeds tostep S434 and performs the processing at and after the step S434.

In the bleeding part determination processing for the bleeding partcandidates extracted as described above, bleeding part determination isperformed based on the color edge feature quantity of the bleeding partedge candidates, similarly to the fifth embodiment.

As described above, in the image processing apparatus 1 of thisembodiment, divided areas 141 which are arranged on opposed two lineswith a divided area 141 to be evaluated as a bleeding part candidatebetween them are used as peripheral areas 143 to be evaluated as ableeding part edge candidate. Thereby, even when a bleeding part isbelt-shaped and only a part of the bleeding part is shown in anendoscope image, the bleeding part can be detected, and the precision ofdetecting a bleeding part is further improved.

In the above fifth to seventh three embodiments, description has beenmade on the case of extracting a bleeding part as a hemorrhage area asan example. However, the present invention is not limited to the aboveembodiments, and various changes and alterations can be made within thescope where the spirit of the present invention is not changed. Forexample, the present invention is also applicable to the case ofextracting a reddened part on the surface of mucous membrane.

INDUSTRIAL APPLICABILITY

An endoscope image captured of a mucosa of a living body by an endoscopeand the like is image processed, and multiple pieces of colorinformation thereof are used to objectively detect a hemorrhage area,whereby assisting a doctor in making medical diagnosis.

The present application is based on the priority of Japanese PatentApplication No. 2005-130229 filed in Japan on Apr. 27, 2005 and JapanesePatent Application No. 2005-130231 filed in Japan on Apr. 27, 2005. Thedisclosure is referred to the description, claims, and drawings of thepresent invention.

1. An image processing apparatus comprising: an evaluation area settingsection for dividing a medical image corresponding to a medical imagesignal constituted by multiple color signals obtained by capturing animage of a living body, into multiple small areas, extracting a smallarea including the outline part of a hemorrhage area from the multiplesmall areas based on at least one of the color signals, setting theextracted small area as a hemorrhage evaluation area, and settingevaluation target areas constituted by the multiple small areas, aroundthe hemorrhage evaluation area; a hemorrhage candidate areadetermination section for extracting hemorrhage edge candidate areasfrom the evaluation target areas based on the amount of change in thecolor signals in the evaluation target areas and determining whether ornot the hemorrhage evaluation area is a hemorrhage candidate area basedon the ratio of the hemorrhage edge candidate areas to the evaluationtarget areas; and a hemorrhage area determination section for extractingthe outline part of a hemorrhage area from the hemorrhage edge candidateareas based on change in two or more of the color signals in thehemorrhage edge candidate areas and determining whether or not thehemorrhage candidate area is the hemorrhage area based on the ratio ofthe outline part of the hemorrhage area to the hemorrhage edge candidateareas, wherein the hemorrhage candidate area determination sectioncalculates a first direction in which the luminance value of the colorsignals in the evaluation target area changes most largely, andcalculates such an evaluation target area that the angle formed by thefirst direction and a second direction from the evaluation target areatoward the hemorrhage evaluation target area is equal to or below apredetermined value, as the hemorrhage edge candidate area.
 2. The imageprocessing apparatus according to claim 1, wherein the evaluation areasetting section calculates the largest value of the amount of change inthe luminance value of the color signals in the small area and settingthe small area as the hemorrhage evaluation area if the largest valueexceeds a predetermined value.
 3. The image processing apparatusaccording to claim 1, wherein the evaluation area setting section setsthe evaluation target areas on a closed curved line surrounding theperiphery of the hemorrhage evaluation areas.
 4. The image processingapparatus according to claim 3, wherein the evaluation area settingsection makes the setting so that the closed curved line surrounding theperiphery of the hemorrhage evaluation areas has a width correspondingto the multiple small areas.
 5. The image processing apparatus accordingto claim 1, wherein the evaluation area setting section sets theevaluation target areas on two lines opposed in a manner that thehemorrhage evaluation areas are positioned between the lines.
 6. Theimage processing apparatus according to claim 1, wherein the hemorrhagecandidate area determination section determines whether or not thehemorrhage evaluation area is a hemorrhage candidate area based onwhether or not the ratio of the number of the small areas calculated asthe hemorrhage edge candidate areas to the number of the small areas setas the evaluation target areas exceeds a predetermined value.
 7. Theimage processing apparatus according to claim 1, wherein the hemorrhagearea determination section calculates the amount of change in theluminance values of two color signals including at least a color signalof G in the hemorrhage edge candidate area, calculates the largest valueindicating that the ratio of the amount of change in the two colorsignals is the largest, and calculates a tentative area used fordetermining whether or not the hemorrhage candidate area is thehemorrhage area.
 8. The image processing apparatus according to claim 7,wherein the hemorrhage area determination section determines whether ornot the hemorrhage candidate area is the hemorrhage area based onwhether or not the ratio of the number of the tentative areas to thenumber of the hemorrhage edge candidate areas exceeds a predeterminedvalue.